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Monterc*.  Caiiiurcia 
ABSTRACT 

It  is  the  purpose  of  these  two  reports  to  demonstrate  with 
very  simple  systems,  several  applications  of  the  optimization  theory 
of  L.  S.  Pontryagin  (1).    An  attempt  to  synthesize  the  control  logic 
using  a  modification  of  dynamic  programming  is  developed,, 

Part  1  presents  the  solution  to  the  minimum  time  problem  and 
the  minimum  fuel  problem  for  a  second  order  system  with  one  zeroc 
The  zero  causes  discontinuities  in  the  state  variables .     Control  dif- 
ficulties are  encountered  when  there  are  discontinuities  in  the  error 
states,  but  these  are  alleviated  by  a  transformation  to  a  system  with 
continuous  variables.    Optimum  control  of  the  transformed  system  is 
then  accomplished  using  the  methods  of  Pontryagin „    The  control 
action  is  then  related  back  to  the  original  plant.    Although  the  in- 
vestigation is  concerned  entirely  with  second  order  systems ,  the 
methods  are  sufficiently  general  to  be  extended  to  higher  order 
systems  with  zeros.     In  part  II  an  example  of  the  extension  is  given,, 
Also  Part  II  presents  various  schemes  for  the  synthesis  of  optimum 
controls . 
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1 .     Introduction. 

When  controlling  the  performance  of  a  system,  it  is  often  desir- 
able to  choose  the  control  that  will  minimize  errors  in  the  system  and 
do  it  in  the  shortest  possible  time.    A  practical  matter  that  must  be 
considered  in  the  optimization  in  relation  to  rapid  action  is  the  fact 
that  control  is  of  a  bounded  nature.     In  a  great  many  important  cases, 
the  constraint  on  the  magnitude  of  the  control  effort  precludes  the  use 
of  classical  variational  techniques  to  design  the  controller. 

In  1956  Pontryagin  hypothesized  his  "maximum  priciple"  which 
has  since  been  proven  a  necessary  condition  for  the  optimization  of 
linear  systems  in  relation  to  rapid  action  /l/.     In  solving  the  mini- 
mum time  problem  for  linear  systems  with  bounded  control,  the 
principle  leads  to  a  "bang-bang"  form  of  control  law.     This  implies 
that  the  control  effort  is  always  being  applied  at  its  maximum  value. 
There  remains,  however,  the  task  of  finding  the  optimum  time  to 
switch  the  control.     Pontryagin1  s  method  leads  to  a  rule  for  switching 
the  controller  which  is  a  function  of  the  initial  conditions  in  the 
system  adjoint  to  the  one  being  controlled.     Generally  these  initial 
conditions  are  difficult  to  find. 

It  is  usually  helpful  to  consider  the  control  problem  using  state 
space  techniques.    The  coordinates  of  the  space  for  an  n      order 
system  here  are  a  displacement  error  and  its  n-1  time  derivatives. 


The  space  may  be  divided  into  two  regions  each  of  which  is  character- 
ized by  the  control  optimal  for  the  trajectories  in  that  region,     Optimum 
switching  between  the  two  conditions  of  the  bang-bang  control  occur 
on  the  hypersurface  dividing  the  space.     The  switching  criteria  can 
then  be  stated  as  a  function  of  the  state  space  variables. 

Of  considerable  value  in  finding  the  switching  surface  is  the  sys- 
tem adjoint  to  the  system.     The  adjoint  can  be  thought  of  as  the  system 
running  in  reverse  time.     By  plotting  trajectories  from  the  origin  of  the 
error  phase  space  "backwards"  in  time,  with  the  control  satisfying  the 
respective  adjoint  variables,  a  surface  is  generated  which  may  be  re- 
lated to  the  optimal  switching  surface  in  the  system  state  space „ 

A  problem  of  interest  occurs  when  the  system  is  of  such  a  nature 
that  when  control  is  applied,  a  discontinuity  appears  in  one  or  more 
of  the  system  states.    This  may  happen  when  the  control  is  of  a  bang- 
bang  form  and  the  forward  transmission  path  of  the  system  contains 
zeros.     It  could  also  show  up  if  the  control  is  of  such  a  form  that  it 
approximates  an  impulse  to  the  system.    When  there  are  discontinui- 
ties in  the  state  space  due  to  switching  it  is  generally  no  longer 
possible  to  write  the  switching  criteria  as  a  function  of  the  state 
space  variables. 

One  alternative  might  be  to  switch  the  control  as  a  function  of 
time.     This  may  be  done  effectively  when  the  number  of  switchings 


to  reach  the  origin  of  the  error  state  space  is  no  more  than  n-1  in  an 
n      order  system.     Such  a  restriction  limits  one  mainly  to  considering 
only  those  systems  with  real,  distinct  eigenvalues.     Large  disturbances 
in  lightly  damped  (oscillatory)  systems  may  require  more  than  n-1 
switchings  to  zero  the  error  states.     The  most  important  consideration 
when  controlling  as  a  function  of  time  is  the  means  of  implementing 
the  switching  logic.     To  accomplish  time  dependent  control,  it  is  vir- 
tually mandatory  that  a  digital  computer  be  inserted  in  the  control  loop. 

Another  approach  to  the  problem  is  to  find  a  system  that  reacts 
identically  to  the  system  with  zeros  except  at  the  points  of  disconti- 
nuity.    Control  of  this  parallel  system  can  be  stated  in  terms  of  the 
state  space  variables.     This  logic  can  then  be  used  to  switch  the 
original  plant. 

This  paper  will  be  an  investigation  into  the  latter  method.     The 
problem  is  as  follows: 

Given  a  second  order  oscillatory  system  with  one  zero, 

find  the  optimum  control  for  zeroing  the  errors  in  the 

system  in  minimum  time  and  for  zeroing  the  errors  with 

minimum  fuel. 

The  method  of  Pontryagin  is  used  to  solve  the  problem.     The  brief 
description  of  the  method  presented  here  is  based  on  the  work  of 
Rozonoer  /l/. 


2.     Pontryagin's  maximum  principle. 

Given  the  system  state  variables  described  by    n    first  order 


differential  equations 

i=l,......,n       (1) 

where    x    is  a  column  vector  in  phase  space  and    u    is  a  column  con- 


*.  =  f.  (x,u,t) 


trol  vector  consisting  of    r    control  elements. 

The  control    u(t)    must  belong  to  a  closed  subset    U    of  admissi- 
ble controls  and  must  be  piecewise  continuous.     The  trajectory    x(t) 
in  the  phase  space  is  uniquely  determined  by    (1)    when  control    ujt) 
and  the  initial  conditions 


x(0)  =  x°  = 


x^ 


x^ 


n 

are  given. 

The  control    u(t)    of  a  system  may  be  considered  optimum  under  a 
variety  of  criteria.    A  large  class  of  optimization  problems  may  be 
solved  by  presenting  the  criteria  in  such  a  way  that  the  solution  is 
attained  by  minimizing  a  linear  function  of  the  final  value  of  the  state 
space  variables.    A  control  must  be  selected  from    U    that  will  trans- 
fer the  system    (1)    from    x°    to  some  fixed  closed  set    G    of  the  phase 


space  such  that 

n+1 

S  =  L     c.x.(T)  (3) 

11 

is  a  minimum.    The  constants    c.    and  the    x     .  coordinate  are  chosen 

1  n+1 

such  that  minimizing    (3)    optimizes  the  system. 

In  a  great  many  cases  optimization  of  only  one  of  the  coordinates 
of  the  system  is  desired.  For  example,  in  order  to  optimize  the  mag- 
nitude of 


T 
F(x(t),u(t))  dt 


"o 
for    T    and    x(T)    either  fixed  or  free  in  a  system    (1)    for    u(t)€U,    a 

new  variable  is  introduced.0 


Vl  =  JtF(s(t)«u(t))dt  (5) 


x°    _  =  0 
n+1 

and  another  differential  equation 

*n+1  =  F(x(t),  u(t)) 
is  added  to    (1).    The  problem  of  optimizing  the  integral  leads  to 
optimizing    x        (T)    at    t  =  T. 

Minimizing    x        (T)    in  the  system    (1)    with    x        (t)  adjoined  is 
accomplished  by  putting  the  problem  in  functional  form    (3)    and  apply- 
ing the  maximum  principle  to  gain  the  solution.    That  is 

n+1 

S=  E    ex  (T)  =*      t(T)  (t) 

i        li  n+1 


is  the  functional  to  be  minimized.    Here  it  may  be  seen  that    c    =  c_ 

.  .  .  ,  c    =0  and    c     ,  =  1 . 
n  n+1 

A  new  dependent  variable    p_(t)    is  now  formed  such  that 


n+1        df   (x,  u,  t) 
P,(t)  =  -Ep    - ^ i=  1, . , n+1      (7) 


i  Is  dx. 


The  function 


n+1 

H  =  S    p   f   (x,  u,  t)  (8) 

.        s  s 

is  introduced  from  which  equations    (1)    and    (7)    may  now  be  written 

dH  dH                                  ,                     ,  .       »_» 

x    = p.  =  - 1=1/ .  ,n+l      (9) 

l       dp.  l           dx. 

l  l 

The  control    u*(t)    is  said  to  satisfy  the  maximum  condition  if 
H(x*(t)  ,p_*(t) ,  u*(t))    reaches  an  absolute  maximum  at  each  time    t 
(0  *  t  ^  T)    where  x*  (t)    and  p_*(t)    are  the  values  of  the  variables  at 
time    t   with    u*(t)  c  U    controlling.    For  linear  systems  of  the  type 

discussed  in  this  paper,  the  necessary  and  sufficient  condition  for 

n+1 
minimizing    S  =  I     c.x,(T)    optimally  with  admissible  control  is  that 

the  control  satisfy  the  maximum  condition. 

To  use  the  maximum  principle ,     H    is  formed  and  maximized  with 

respect  to    u(t) .    This  produces  a 

u*(t)  =  0  (x,  pj  (10) 

which  may  be  used  with  equations    (9)    and  the  boundary  conditions 

to  find    _u*(x).     If  the  end  point  of    x(t)    is  not  fixed,  it  becomes 


necessary  to  obtain  boundary  conditions  on   p_(t)    in  order  to  arrive  at 
a  solution.    The  conditions    p_(T)    may  be  found  using  a  function 
F(x)  ^    0    which  describes    G    and    x   (T)  eG,  the  end  point  of  an  opti- 
mum trajectory.    The  form  of   p_(T)  will  be  stated  without  detailed 

explanation;  however,  it  may  be  noticed  that  at  time    t  =  T,  p_(T)    is 

n+1  1 

orthogonal  to  a  hyperplane    L    a.(x.  -  x, )  =  0  through  the  endpoint  of 

the  trajectory  and  directed  toward  that  portion  of    G    where 

n+1  n+1         x 

S     c.x,  ^    £     c.x,  (T).     The  coefficients    a.    may  be  expressed  as  a 
1111  l 

1 
linear  combination  of  the    c.    and    b.(x   (T)),  the  latter  being  coeffi- 
cients of  a  hyperplane  through    x   (T)    bracketing    G. 
Thus 

p.(T)  =  -Xc.-Mb.(x1(T))  (11) 

where    X   and    fi  are  non-negative  numbers  one  of  which  may  be  set 
equal  to  unity  as  it  is  only  the  ratio  that  is  important. 

Generally,  three  situations  arise  as  to  final  boundary  conditions. 

(i)        If    x,(T)    are  specified  for    i=l„2, ,m    then  these 

become  the  boundary  conditions  for    (9). 
(ii)       If    x.  (T)    are  internal  points  of    G    for    i  (1  £  i  *  n+1) 

then    b.fac  (T))  =  0    and    p.(T)  =  -c. 
l^-  i  i 

(iii)      If   x.  (T)    are  boundary  points  of    G    for  some    i  (1  £  i^n+1) 

then    F(x(T))  =  0    and  the    p.(T)    are  as  in    (11). 
When    F    is  differentiable ,  the  bracketing  hyperplane  through 


x      has  coefficients 


i^-  ox. 

1 


x.  =  xi1(T)  (12] 


If  finding  the  optimum  control  for  minimum  transit  time  another  condi- 
tion must  be  fulfilled  since    T    is  not  fixed  beforehand.     This  condition 
is  that    H(T)  =  0. 

3  .    Development  of  system  equations. 

The  equation  of  a  second  order  system  with  zeros  may  be  written 
c  +  2Cwc  +  a>sc  =  a   u  +  a0u  (13) 

where    c    is  the  output  variable  of  the  system  and    u    is  the  output  of 
a  controller. 

This  paper  is  concerned  with  control  of  similar  systems  that  are 
purely  oscillatory  in  nature  i.e.     C  =  0.    To  facilitate  ease  of  com- 
putation in  the  analysis, equation    (13)    is  scaled  to 

c  +  c  =  a   u  +  u  (14) 

which  when  written  in  terms  of  the  Laplace  transform  of  the  output 

variable  becomes 

(a   s+  l)U(s) 

C(s)  =  — l— (15) 

s    +  1 

This  system  is  represented  in  block,  diagram  form  in  Fig.   1, 


Fig.   1  -  Block  diagram  of  control  system 
The  response  of  the  system  to  a  step  input  is  investigated  more 
readily  by  means  of  the  error  variable 

e  =  r  -  c  (16) 

If  the  input    r    is  fed  forward  as  in  Fig.  2,  the  Laplace  transform  of 
the  error,  given 


c(0)  =  c° 
6(0)  =  c° 
R(s)     =     r0/s 


(17) 


becomes 


E(sj  =     (r0  -  c°)s  -  (c°  +  airQ)  -  (axs  +  l)U(s)         ^ 


ss  +  1 


8) 


Now  the  problem  of  zeroing  the  error  states  reduces  to  that  of  zeroing 
the  error  initial  conditions  in  the  system. 


^ 

e 

CONTROL 

Ym 

u 

ans  +  1 
s3+l 

c 

V- 

rxD 

s 

Fig.  2  -  Controlled  system  with  input  fed  forward 
Finally  with  the  introduction  of  state  space  variables 

ei  =    -e 


(19) 


the  system  equations  can  be  written  in  vector  matrix  notation  /3/ 


e  = 


1-                - 1 

[—            —i 

0      1 
-1    0 

_                 _ 

e  + 

0 

aid  +  u 

_ 

(20) 


4.    The  minimum  time  problem. 

The  problem  is  stated  as  follows: 

Given  the  system    (20)    and  a  control  force  of  bounded 
magnitude     |u|  ^   1,  find  the  optimum  control    _u*(t)    to 
transfer  the  state  variables  from  some  initial  point  in 
the  phase  space  to  the  origin  of  the  phase  space  in 
minimum  time    T. 


10 


That  is ,  given 

e(0)  =    e° 
e(T)  =     0 

|u|    ^     1 

and  the  system    (20),    find    u^(t)    such  that 

T 
S  =  f  adt 

is  a  minimum  where    a.     is  a  positive  constant, 


Introduce 


n+ 


.  =  e3  =  S  =  f   adt 


The  system  equations  then  become 


e  = 


~0    1     0~ 

-10    0 

e  + 

_0    0    0_ 

0 

ai 

u 

+ 

u 

a 

(21) 


(22) 


(23) 


(24) 


Because  of    (21)  ,  the  functional 

3 
S  =  E  ce  (T)  =  c3e3(T)  (25) 

1 
and  since  we  wish  to  minimize  this,    c3  =  1    is   chosen.     e3(T)  is  not 

limited,  hence  the  boundary  condition  becomes 

PaCT)  =  -c3  =  -l  (26) 

By    (8),  the  hamiltonian  becomes 


H  =  Pie2  -  psei  +  p2(aiu  +  u)  +  p3a 


(27) 


Since    p3  = 


-SH 
9  e3 


=  0    it  is  evident  that    p3     is  a  constant  and  there- 


fore   p3  =  p3(T)  =  -1    and  now    H    is 


11 


H  =  Pie2  -  psei  +  Psfeiti  +  u)  -  a  (28) 

which  is  maximized  in    u    if 

aiU+  u  =  N[sgn  p8]  (29) 

where    N  =  max  |aiu  +  u  |  f or  each  fixed    t(0  ^  t  £  T) .    The  control 
u*(t)    which  satisfies  these  conditions  is  a  "bang-bang"  type  control 
where    u  =  ±  1    at  all  times  and    u    at  the  moment  of  switching  is 
unbounded . 

Since    e3    has  served  its  purpose  in  the  optimization  process, 
we  may  now  return  to  the  second  order  system  and  solve  for  the 
"impulse"  variables.    By    (9), 


£  = 


0      1 
-1      0 


(3  0) 


and  the  solution  for    u*(t)    becomes 

u*(t)  =  1  •  sgn[cos(t +0)]  (31) 

where    ©  is  a  phase  angle  dependent  on    x°. 

Several  properties  of  the  optimum  controller  are  now  known. 
First,  the  control  is  a  bang-bang  type  which  applies  maximum  effort 
at  all  times  in  one  of  the  two  "directions".     It  is  switched  periodi- 
cally from  one  state  to  the  other  every  half  cycle  until  the  origin  is 
reached.     Notice  that  each  time  the  control  is  switched,  a  disconti- 
nuity appears  in  the    e2    variable.     This  occurs  because    u     contains 
an  impulse. 


12 


+  + 

A  es  =       (-e!  +  aiu  +  u)  dt  =      axudt  =  ax  [u(t  )  -  u(t  )  J  (32) 

«!■£-  j£-  S  o 

S  S 

where    Ae2    is  the  discontinuity  in  e%    at  the  time  of  switching  t    . 

One  would  now  like  to  find  a  switching  curve    L(e)    which 
divides  the  phase  plane    ei  vs  e%    in  such  a  manner  that  control 
u*  =  +1    is  optimum  in  the  space  to  one  side  of  the  curve  and    u*  =  -1 
elsewhere.     Control  would  be  switched  when  the  trajectory    e_*  (t) 
crosses  the  curve.    The  discontinuity    Ae2    precludes  this  possi- 
bility.    For  example,  examine  the  trajectory    e_*(t)  for  some  initial 
conditions  that  dictate    u*  =  -1    for  optimum  control.    At  the  point 
where  this  trajectory  crosses    L(e)    the  optimum  becomes    u*  =  +1. 
The  control  switches  and    Aes  =  +2a!    occurs  which  places  the  states 
back  in  the  space  where    u*  =  -1  was  optimum.     Here  the  control 
switches  again,    Aez   = -2ai    occurs  and  chatter  motion  begins.    The 
fact  that    e%     is  multiple  valued  at  the  instant  of  switching  makes  a 
simple  realization  of    L(e)    impossible. 

For  periods  between  switchings  where    u  =  0,     the  system  is  well 

behaved  with  the  solution  for  the    k        interval 

ej.(t)    =    K  cos  (t  +  0ic)  -  6 

(33) 
es(t)  =    K  cos  (t  +  0k  +  it/2) 

where    6=1*  sgn    p2    and    K,0k    depend  on  conditions  of  states  at 

the  start  of  the  k        interval. 


13 


4. 1    The  transformed  variable. 

The  search  for  a  variable  of  the  system  on  which  to  control  leads 
to  the  possibility  of  "subtracting  out"  the  discontinuity  present  in 
e2    at  times  of  switching. 

The  Laplace  transforms  of  the  system  variables  are 


P  ,  .        efs  +  ej  +  frig  +  1)  U(s) 

Ei(s)  = sn 


„   U\       egs  -  ef  +  sfajS  +  1)  U(s) 
£a(s)  = 3— — 

s    +  1 


where 


which  for  any  instant  of  time    t  (0  ^  t  <  ti) 


(34) 


U(s)  =  6(—  -  —  e  tlS+  —  e  t3S-  .......)  (35) 

s         s  s 


(37; 


U(s)  =  -|-  (36) 

Equations    (34)    then  become 

P  (os       e?  s2  +  (eg  +  aid)  s  +  6 
Ei(s)  = 

s  (s3  +  1) 

Ea(s)  =  fei  +  ai6)  s  +  (~e°  +  6) 
s2  +  1 

By  means  of  the  initial  value  theorem,  it  is  seen  that 

lim  ei  (t)  =  lim  sEi  (s)  =  e° 
t-0  s^» 

lim  e2(t)  =  lim  sE2(s)  =  eg  +  ax6 
t-»Q  s-*00 


(38) 


At  time    t  =  0,  e2     jumps  to    e2  +  ai6  .     To  remove  this  discontinuity 


14 


consider  the  transformed  variables 
Y*(s)  =  Ei(s) 
Y2(s)=E2(s)-  ^ 

By  virtue  of    (39)    and    (20) 


(39) 


sYiCs)  =  sEi(s)  =Ea(s)  =  Y2(s)  + 


ax6 


sY2(s)  =  sE2(s)  -  ai6  =  -Yx(s)  +  -j 


or 


Y.  = 


0      1 
-1      0 


Y.+ 


1 


(41) 


where    6    is  a  unit  step  function  with  sign  to  be  determined.    The 

system    (41)    is  identical  to  that  of    (2  0)    except  for  the  action  at 

time  of  switching.     It  should  be  noted,  however,  that  care  must  be 

taken  in  assigning  final  values  to  the  system  described  by    (41)    if 

the  two  plants  are  to  be  controlled  in  parallel.    The  final  value 

theorem  and    (39)  gives 

lim  y2(t)  =  lim  sY2(s)  =  lim  s(E2(s)  -  ^- )  =  -a^  (42) 

t-»  s^O  s^O 

From  this  it  is  observed  that  zeroing  the  final  states  in    (20)    is 

analagous  to  zeroing    yx  (T)    and  attaining  a  final  value 

y2(T)  =  -ax6  (43) 

in  the  system    (41). 
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4.2    Boundary  conditions  and  final  control. 

From    (38)    and    (39)    it  is   clear  that  the  initial  conditions  on 

the    e    and    y    variables  are  identical.    From    (39)    it  is  also  seen 

that 

Yi(T)  =  ex(T) 

(44) 
y2(T)  =  ea(T)  -  ai6(T) 

At  this  point  in  the  pursuit  of  the  optimum  control,   it  becomes  neces- 
sary to  investigate  the  system  action  possible  at  time    t  =  T    under 
admissible  control.    Ae2    of    (32)    provides  a  means  of  changing  the 
value  of    e%    instantaneously  by  an  amount  dictated  by  the  constraints 
on    u(t).    With  this  in  mind,  it  is  noted  that  appropriate  use  of    Ae2 
within  the  bounds  of  allowable  control  may  zero  the    e2    variable  in 
zero  time  given  that    es(T)    is  within  range.     The  conditions    (21) 
and    (44)    with    (32)    indicate  that  for 

|y3(T)|  ^  ax  (45) 

the  system    (20)    may  be  zeroed  instantly   .    The  boundary  conditions 
on    (41)    then  become 

y.(0)  =  yj  =  e°  1-1,2 

Yx(T)  =  0  (46) 

|y2(T)|  ^  ax 


The  conditions  are  stated  in  terms  of  the    y    variable  for  con- 
venience in  order  that  notational  problems  arising  from  multiple 
value  of    e2(0)    be  avoided. 
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The  final  controller    u2(T)    that  must  zero  the  errors  for    t  >  T 
has  two  conditions  imposed  upon  it  i.e.  , 
aiu2  +  u2  =  0 


u2(T)-5(T)  =  - 


(47) 


The  solution  to    (47)    is 

u3(t)  =    "yaz(T)       exp    (  -^i_ )  t  *  T  (48) 

ai  ai 

It  is  assumed  then  that    u2  (t)    is  available  at  time    t  =  T    so  that  the 

boundary  conditions  on  the  system  are  as  stated  in    (46). 


4.3    Switching  functions . 

The  method  of  finding  a  function    L  (y_)    with  which  to  describe 
the  switching  criteria  for  the  optimum  trajectory  proceeds  as  follows. 
As  in  the  discontinuous  case,  it  is  desired  that 

rT 

S=       adt  (49) 

be  minimized,  therefore,  another  variable    y3  =  S  =  c3y3(T)    is 
adjoined  to  the  system  and  once  again    Cx  =  c2  =0.    The  hamiltonian 
becomes 

H  =  Piy2  +  Piax6  -  p2yx  +  p26  -  a.  (5  0) 

This  is  maximized  in    6    when 

6   =  1  •  sgn  (axpi+  p2)  (51) 

With  this  control,  trajectories  are  circular  about    (6,  -a^)  with 
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radius  determined  by   y_°.     (See  Fig.  3) 


6  =  -1 

-  6  =  +l 

>  direction  of 

positive  time 


Fig.  3     yi  vs    ys    phase  plane  with  trajectories  for  6  =  ±  1 
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Previous  arguments  have  determined  that  the  conditions  on  the 
system  are 

y^O)        =     y°  i=  1,2,3 

yi(T)         =     0 

|y2(T)|     *     ax  (52) 

H(T)  =      0 

PaCD        =     -i 
The  function    F  =  —  (yl  -  af )  ^  0    may  be  used  to  describe    G. 


From  this 

dF 


ba(yi(T))  = 


dy2 
and 


=     yi  (53) 

ys  =  Yz 


pa(T).=  -Xc2  -  pb»to(D)  =  -HY2  (54) 

where    /i  *>  0    with  modulus  such  that    F(T)  =  0.     In  the  phase  plane 
of    Pi  vs  P2    it  is  sufficient  to  note  that  for  trajectories  terminating 
at   y2(T)  =  -ai,  p2(T)  *  0    and  for  trajectories  ending  at   ya(T)  =  +ar 
P2  (T)  ^  0.    This  information  in  addition  to  the  control    (51)    com- 
pletely define    L(y_)    for  trajectories  ending  on  the  extremes  of  the 
line  segment     |y2(T)  |  *  ax. 

Fig.  4    depicts  representative  action  for  optimum  trajectories 
terminating  at   y2(T)  =  -a*,  yi(T)  =  0.    Trajectories  ending  at 
y3(T)=+ai,  yi(T)  =  0    are  mirror  images.    The  optimum  switching 
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curves  are  generated  by  picking  arbitrary  values  of   p_(T)    from  the 
admissible  set  for  the  corresponding  boundary  values  of  y_(t)    and 
working  backwards  in  time  plotting  the  switching  points  determined 
from   p_(-t)    on  the    yi  vs  Yz    phase  plane. 
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For  trajectories  ending  in  the  interior  of  the  line  segment  where 
|y2(T)  |<  ai  ,  b2(y12(T))  =  0    and,  therefore,    p2(T)  =  0o    This  com- 
pletes the  information  necessary  to  describe    L(y_).    Fig.  5    shows  a 
representative  trajectory  arrived  at  by  translating  switching  criteria 
from  the   p_  -  plane  to  the   y_  -  plane.    Fig.  6    portrays  the  curve  with 
all  dimensions. 


switching   curve 


Fig.  5  Pi  vs  p2    and  yx  vs  y2  phase  planes  with  complete  switching 
curves 
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Fig.   6    -    Optimum  switching  curve  (  minimum  time  ) 
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Fig.  7    -    Optimum  trajectories  of  e_(t)  and  y_(t)  (minimum  time  ) 
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switching  curve 


trajectories 


Fig.  8    -    Optimum  trajectories  of  e_(t)  and  y_(t)  (minimum  time) 
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Fig.  9    -    Optimum  trajectories  of  e_(t)  and  Y.(t)  (  minimum  time  ) 
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5  .    The  minimum  fuel  problem. 

The  minimum  fuel  problem  is  solved  by  minimizing  the  integral 

rT 
J=       (|u|  +   |alU  |)dt  (55) 

Jo 

in  the  system    (20)    where    T    is  not  specified.     It  appears  simpler, 

however,  to  once  again  make  use  of  the  transformed  variable    y_(t) . 

By  minimizing 

T 
J=  [    |u|dt  (56) 

in  the  transformed  system    (41),  the  desired  result  can  be  obtained 

provided 

i)       the  switchings  in  the  time  interval    0  ^  t  ^  T    are 

kept  to  a  minimum. 

ii)     adjustment  is  made  at  time    t  =  T    when  fuel  is 

consumed  zeroing  the  error  states    e_(t)    with  the 

exponential  control    u2(T). 

After  adjoining    (56)    to  the  system    (41),  the  hamiltonian 

becomes" 

H  =  Piy2  -  p2yi  +  u(aiPi  +  pa)  -   |u|  (57) 

Since    T    is  not  specified    H(t)  =  0.    With    u(t)    constrained  as 

before,  the  control  that  maximizes    H    with  respect  to  _p_(t)  is: 

u*  =  1  •  sgn  (aiPi  +  p3)  |axPi  +  p2  |    >  1 

(58) 
u*  =  0  |aiP!  +  p2  |    <   1 
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5. 1    Initial  Conditions. 

Taking  the  time  derivative  of    H 

dH        ,  x    du         d  I  u  I  ,__. 

—  =  (aiPl  +  Ps)   ~dt     "      dt  (59) 

it  can  be  seen  that  — —  =0    if    ~  =  0.     It  may  also  be  argued  that 

dt  dt 

the  change  in  the  hamiltonian  with  time  is  zero  if 

d I u I  A lul 

am  +  Ps  -  ^L-l    -  -jl-J  (60) 

Since    u(t)    is  switching  between    u=  0    and    u=  ±1    and  vice  versa 

this  means  that  the  hamiltonian  remains  constant  if  the  control  is 

switched  at   ajPx  +  p2  =  1  •  sgn  (Au) ,   (See  Fig.   10). 


u*  =  1 


1P1  +  Pa  =  1 


axPi+  p2  =  -1 


curve 


Fig.  10    Switching  criteria  in  px  vs  p2  phase  plane 
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By  choosing  control     u*(t)    the  hamiltonian  remains  at  its  maxi- 
mum value  i.e.  identically  zero  from  time    t  =  0      after  initial  control 
has  been  applied  until  time    t  =  T.     This  control  minimizes  the 
integral    (5  6)    but  does  not  necessarily  minimize   total  fuel  when  fuel 
consumed  at  switchings  is  added.     In  order  to  minimize  switchings, 
it  appears  necessary  to  choose  the  degenerate  case  i.e.     u  =  0 
until  such  time  as    aiPi  +  ps  =  1   •   sgn  (Au)    where    Au    is  the  change 
in    u(t)    when  turning  the  control  on.     Notice  that  this  choice 
guarantees  that    H(t)  =  0    for  all    t,   0  ^  t  £  T.    With  this  in  mind, 
the  problem  remains  to  minimize  fuel  in  the  non-degenerate  case. 
For  this  purpose  it  will  be  considered  that  time    t  =  0    is  that  time 
when 

aiPi  +  p2  =  1   •   sgn  (Au)  (61) 

and  initial  control  is  applied. 

At    t  =  0    it  may  be  verified  from    (61)    and  because    H  (0)  =  0 
that 

Pi°y§  -  Pi  x?  =  0  (62) 

5  .  2    Final  boundary  conditions. 

In  order  to  investigate  final  value  boundary  conditions,  the 
optimum  trajectories  terminating  such  that    yx(T-At)>0  are   consi- 
dered.    Trajectories  in  the  rest  of  the  space  are  mirror  images.    As 
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in  the  minimum  time  problem,  an  optimum  trajectory  terminating  at 
Ya(T)  =  -ai,yi(T)  =  0    is  investigated  first.    The  determination  that 
Pa(T)  ^  0    as  argued  in    (54)    is  still  valid.    This  condition  on    Pa(T) 
along  with  the  fact  that    H(T)  =  0    precludes  the  possibility  of  a 
trajectory  terminating  as  above  with    u(T)  =  -1.    The  following  cases, 
however,  do  apply.     Consider 

H(T)  =  -diPitt)  +  u(T)  [axPi(T)  +  pa(T)]  -    |u(T)  |   -  0  (63) 

This  condition  implies  that  if    u(T)  =  0    then  px(T)  =  0    and  if 
u(T)  =+1    then    Px(T)  *  0    and  p2(T)  =+1.    Fig.   11    portrays  the  locus 
of  admissible  points    jd(T)    and  the  switching  curves  generated  by 
these  criteria  in  the    yivsys       phase  plane  are  as  in  Fig.  11a. 


admissible 
points  pjT) 


'  Fig .  11    Admissible  points  p_(T)  for  y3  (T)  =  -ax 
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Fig.  11a    Switching  criteria  for  y3  (T)  =  -a!    (minimum  fuel) 
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Optimum  trajectories  terminating  on  the  line  segment    yi(T)  = 
0,    |y2(T)  |<  ai    must  be  investigated  in  a  fashion  similar  to  that 
used  with  the  minimum  time  problem.     Since  a  final  boundary  point 
y2(T)    is  not  fixed,  we  may  substitute  a  final  condition  on    P2(T) 
to  reach  a  solution.    At  this  point  it  becomes  necessary  to  decide 
on  the  final  value  functional  to  be  minimized.     It  is  first  noted 
that  if  the  final  control  to  the  line  segment  is    u(T)  =  0,  then 
-a1<y2(T)  <0.     (It  must  be  remembered  that  investigation  is  of 
trajectories  such  that    y^T-At^  0).     If    u(T)  =  0    then  also  ys(T)  = 
e2  (T)    and  in  order  to  minimize  the  fuel  consumed  by    u2(T)    to  zero 
e2    after  time    T    then     |e2(T)|=   |y2 (T)  |    must  be  minimized. 

If  the  final  control  is    u(T)  =  -1    (u(T)  =  +1    is  not  possible  for 
trajectories  terminating  on  this  side  of  the  line  segment)  then 
©2(1")  =  y2(T)  -  ai    and,  therefore,    |y2(T)  -  ai  |   must  be  minimized. 
In  both  of  the  above  cases,  it  may  be  seen  that    y2(T)    must  be 
maximized  on  the  line  segment  in  order  that  fuel  consumed  by 
u2(T)    to  zero  the  error  states  be  minimized.    Therefore,  the  functional 

to  be  minimized  is 

3 
S  =  Ley  (T)  =  -y2(T)  +  y3(T)  (64) 

1 
where 

pt 
y3(t)  =        |u|dt  (65) 

By  prior  arguments    p2(T)  =  -c2=+l    and    p3(T)  =  p3(t)  =  -c3  =  -1 . 
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5.  3    Generating  the  switching  curve  segments. 

It  is  now  helpful  to  look  at  the  hamiltonian  under  each  of  the 
above  conditions,  i.e.  ,     u(T)  =  0    and    u(T)  =  -1.    In  the  first  case 

u(T)     =     0 

H(T)     =     H(t)  =  0 

yi(T)    =     0  (66) 

Pa(T)  =    +1 

-ai<y2(T)<0 
and 

H(T)  =  P!(T)y2(T)  =  0 
which  implies  that    Pi(T)  =  0.    Fig.   12      shows  the  switching  gener- 
ated by  this  condition. 

ys 


/ 


/ 

'u-0      ,' 
/ 
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+ 


i 

i     ; 

|U=+1 

1       I 


-1 


-ai- 
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\      \ 
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u=0 


/ 


/ 


Yi 


Fig.  12    Switching  criteria  for  u(T)  =  0  (minimum  fuel) 


29 


Next  is  considered  the  case  where 


u(T) 

= 

-1 

H(T) 

= 

H(t) 

Yi(T) 

= 

0 

P3(T) 

= 

+  1 

=  0 


(67) 


and 


from  which 


H(T)  =  Pi(T)y2(T)  -  l[alPl(T)  +  1  ]  -  1  =  0 


Pi(T)  = 


ya(T)  -  ai 


:6s: 


Since    u(T)  =  -1    and  p2(T)  =  +1,  conditions    (58)    are  met  only  when 

-2 

Pi(T)  <  which  implies    y2(T)  >0.     In  Fig.   13    these  trajectories 

and  switching  curves  are  plotted. 


admissible 
points  p_(T) 


Fig.  13    Admissible  p_(T)  where  u(T)  =  -1 
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5  .4    The  complete  switching  curve. 

Because    T    was  never  specified  and  because  the  fuel  consumed 
at  switching  was  handled  as  a  side  condition,  a  composite  of  all  the 
calculated  switching  curves  indicates  areas  in  the  phase  plane  where 
criteria  for  optimum  control  appear  contradictory.     In  these  areas, 
analysis  by  graphical  means  or  actual  computation  will  clear  up  the 
situation.    Fig.  14  depicts  the  composite  of  the  first  two  criteria 
analized. 


u  =  +l 


Fig.   13a    Switching  criteria  where  u(T)  =  -1 
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In  Fig.   14,  region   A    is  an  area  where  there  is  a  question  con- 
cerning whether  it  is  optimum  to  switch  for     |y2  (T)  |  =  ax    or 
|y2(T)  |  <  ai.    By  graphical  analysis,  it  may  be  seen  that  it  is  opti- 
mum to  switch  so  that     |ys(T)  |  =  ax. 

A  similar  contradiction  between  trajectories  switching  for 
0  <  ys(T)  <  ai    and  -a!<y2(T)  <  0    may  also  be  resolved  graphically. 
The  final  result  consisting  of  switching  criteria  to  zero  the  errors 
in  the  system    (20)    with  minimum  fuel  is  given  by  Fig.   15. 


Fig.   14    Region  of  conflicting  optimum  criteria 


*Appendix    I    presents  computational  analysis  of  the  resolving 
process. 
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Au=0 


4iu=0 


Fig.   15    Switching  criteria  for  minimum  fuel,  ax  =  1 .  0 
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6.    Realization  of  control. 

With    L(y_)    providing  the  switching  logic  everywhere  except 
where    yi(T)  =  0    and     |ys(T)  |  <  ax    and  with    u2    available  for  final 
action,  the  controller  can  now  be  built  (theoretically). 


Fig.  16    Block  diagram  of  the  controlled  plant 
By  subtracting    ai6    from    e2  (t)  „  the  variable    y»(t)    is  gener- 
ated for  use  in  control  logic    (Fig.  16).    At  time    t  =  T,  Sw  2 
designates    us    as  the  control  and  errors  are  instantaneously  zeroed, 
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The  complexity  of  the  logic  necessary  to  implement  the  switch- 
ing criteria  may  lead  one  to  desire  a  simpler,  quasi-optimum  control „ 
Two  such  controls  are  pictured  in  Fig.   17  for  minimum  time  problem „ 
Although  detailed  investigation  of  these  controls  was  not  carried 
out,  it  is  submitted  that  both  controls  are  close  to  optimum  especially 
for  large  initial  disturbances  in  the  error  space.    Both  controls  were 
designed  with  the  thought  that  chatter  motion  would  not  be  tolerated, 
switching  function  would  be  linear  over  a  large  range,  and  exponen- 
tial control  was  available  at  end  point  of  trajectory. 

It  is  further  suggested  that  the  system  might  be  controlled  on 
the  error  states  instead  of  the  transformed  variable  if  constraints  are 
put  on  time  intervals  between  switchings  so  that  chatter  motion  is 
avoided. 
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7.    Conclusions. 

The  methods  used  in  this  paper  to  arrive  at  a  solution  may  be 
used  to  good  advantage  in  the  investigation  of  any    n       order 
system  with  no  more  than    n-1    zeros.    The  maximum  priciple  pro- 
vides a  powerful  tool  in  optimization,  particularly  for  linear  systems. 
Often  the  method  of  Pontryagin  will  indicate  areas  of  interest  to 
investigate  when  searching  for  an  optimum  control  even  if  the  unique 
solution  is  not  readily  forthcoming. 

The  problem  of  controlling  a  plant  with  zeros  is  analagous  to 
controlling  a  plant  without  zeros  using  an  impulse- step  type 
controller.    Results  obtained  in  this  paper  can  be  adapted  to  formulate 
the  logic  of  this  type  control. 

The  realization  of  the  true  optimum  switching  logic  in  a  practical 
system  may  in  many  cases  not  be  worth  the  effort.    Quasi-optimum 
control  using  simple  switching  functions  that  are  for  the  most  part 
linear  is  a  subject  for  further  investigation.     Settling  time  for  the  sys- 
tem is    relatively  insensitive  to  limited  variations  from  the  optimum 
when  trajectories  are  out  beyond  the  first  cusp  of  the  switching 
curve.    The  two  quasi-optimum  controls  suggested  in  this  paper 
concentrate  on  avoiding  chatter  motion.     It  may  be  that  a  control 
using  "controlled  chatter  motion"  /2/  would  be  acceptable  in  parti- 
cular systems .    This  type  would  be  particularly  attractive  for  control 
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of  trajectories  in  the  area  of  the  first  cusp  when  linear  control 
functions  are  desired. 
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Appendix  I 

Graphical  analysis  of  optimum  criteria  interface 
Given   y_°   in  region    B    of  Fig.   16    the  problem  is  to  find  which 

of  the  two  possible  switching  criteria  is  optimum,  i.e.  ,  whether 

for  trajectories  such  that    yi(T  -  At)>0  it  is  optimum  to  switch  for 

0<y2(T)<a1    or   -a1<y2(T)<0. 


Fig.  16     Region  of  conflicting  switching  criteria 
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Consider  ;y0   on  curve    1    of  Fig.  16    a  distance    R    from  the 

origin.    Does  it  take  less  fuel  to  apply    u{0)  =  -1    and  zero  the 

corresponding  values  of    e_   or  should  the  degenerate  case  be 

chosen  until  such  time  as  curve   2    is  reached  (where    y{  =  0    and 

Yh   =  R)  and  then  switch?     The  fuel  consumed  when  switching  from 

curve   2     is 

ai  when  control  is  turned  on 

T1-  t/2 
tt/2  for    I     |u|dt 

a.i  when  control  turned  off  at    yh  -  0 

|y2  (T)  |    to  zero  corresponding    e2  (T) 

which  when  totalled  is  equal  to  R  +  tt/2  „ 

The  fuel  consumed  when  switching  from  curve  1   is 

ai         when  control  is  turned  on 

rT 
T  for  |u|dt 

l^i  ~  ys(T)  |    to  zero  corresponding    e2(T) 

The  time    T    which  is  the  time  for  the    y_(t)    to  progress  from    y_° 

at  curve    1   to  the  line  segment  may  be  portrayed  by  use  of 

as  in  Fig.   17. 
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d]Pi   +  Pa  =  1 


SLlPl    +   P2=  "I 


Fig.   17    Time  T  as  a  function  of  0X  and  02  in  £  plane 


Here  it  is  remembered  that 


and  that 


which  implies 


Pl(T)        =  y^TTi 

Pa(T)  =  +1 

aiP?  +  Pi  =  -1 

T  =  01  +  02  +  ff/2 


H(0)  =  Pl°yi  -  pgy?  =  0 


Pi0  =  Yi° 


:69: 


Pa       ys 
The  total  fuel  from   l    then  is    ax  +  0X  +  02  +  tt/2  +   |ax  -  y2 


and 
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the  angles    0i    and    02    may  be  related  to  points  in  the    yi  vs    ys 

space. 

0!  -  tan      -  -— ^-  =  tan       -  -—5- 
Ps  y2 

^_   -  t^"1        Pa(T)    _  +     -1    ai  ~  Ys 
03  =  tan      -  -r-r  =  tan 


(71! 


Pi  (T)        u""  2 

The  point  at  which  equal  fuel  is  consumed  when  switching  on  curve 
1  or  curve    2     is  that  point  where 

R*  =  23l  -  y2(T)+  0!+  03  (72) 

For    R  >  R*    optimum  switching  is  on  curve   2     . 
Computation  of  an  example  in  the  case  where    ax  =  1.0  revealed^ 


yf       = 

-1.355 

Y%        = 

2.140 

yx(T)  = 

0.0 

yaCT)  = 

0.347 

R* 

2.533 

Fuel    =     4.104 
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